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(57)Abstract: f 
PROBLEM TO BE SOLVED: To generate the video of an 
AR game based upon the subjective viewpoint of a 
player and to generate the whole image of the AR game 
viewed from an objective viewpoint and the video of the 
AR game based upon a viewpoint that an observer 
desires. 

SOLUTION: In the figure, a part 201 is a game state 
management part and manages the state of the AR 
game (information regarding the rendering of a virtual 
body 102, the score of a player 101, the number of 
rounds of the AR game, etc.). A part 202 is an objective 
viewpoint video generation part and generates the video 
of the virtual body 1 02 viewed through a camera 1 03. A 
part 203 is an objective viewpoint video composition part 
and generates the composite video of the video of the 
virtual body 1 02 and real video and outputs it to a 
display 106. A part 212 is a subjective viewpoint video 
generation part and generates the video of the virtual 
body 102 viewed from an HMD 107. A part 213 is a 
subjective viewpoint video composition part and generates the composite video of the video of 
the virtual body 102 and real video and outputs it to the HMD 107. 
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-^ffi!ta^2 0 4 HMD 1 0 7<OSjS<ajB3B» 
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1 0 0 5 6 ] Xr "/7S 8 0 2 IC8WC , $v-4ttJ** 
3*2 0 lid*, y-AtfXRJR 1 0 20iWffc 

[0057]Xf "/7°S 8 0 3 (CfcWC, 

&j«S2 o 2 t£mmmam&m 1 2#. 

t mwmmm^&i-t : . 

[ 0 0 5 8 ] 'y7°S 8 0 4 fcJaWT , £j|$£ftfll 
£$SI5 2 0 3 i:±fS?MWft^gB2 1 3*\ 
HMD107fc#.*5103 Ttt(RSfi^SI9WRt A 

[ 0 0 5 9 ] Xf «y 7 S 8 0 5 fcfcWt , SIHSAttft 
■&jfifflf2 0 3 fc±f$*MWfti=riS*2 1 3*>\ fix-eft 
A RUM i: ittffljS A RWrnZZLffrfl . 

[ 0 0 6 0 ] Xf 7 7S 8 0 6 ttJV vr , gftflARft 
^«*2 0 3fc±«aSW«^J«»2-l 3*\ 
SWMARBMf^T^TM 1 0 6(C, ±iWMA 
RWRtHMDl 07lz£f)i-h. 

[006 1] a±««aB:AR4*-A*WT<-*4 , r« 

[0062] utcommfc. m-&mmjk*um. 

HMD 1 ■Q7*fflVVfcAR?«-A-fc*»V|T, 7V—V1 
0 1 jyfl^E#fc*MSjS A Rlftftt#m £ fe** 
T* « . 

[0063] [®2«»u $ i ommmmn, 

HMD 1 0 7«:t'T^:-'-*/l'-^;M7V)HMD*C* 
ofeW, HMD1 O7te%fri'-X)\>-0)?47-Cfoi 

xi>ru—*n o lttARy-Atfrdik*^**. 

[0 0 64] 01 3lt%&i'-ZH^0>f4 70)HMD 
13 0 1 *jptB| , Cft6. ifl:*J|a|iattJ«-6HMD 1 3 
OHiWT'*^ HMD 1 3 0 misldfcfcvv^l 

[0 0 6 5] 130 1H3^S'-X/l^*>?-f7V>HM 
DX\ 1 3 0 2li7V~ VI 0 1 cO@T"J>!>. 
[00 66] i^iM5 0 1 CfcMSSfttt 1 0 2Dlftft 

wmmu 1 3 0 2mw.frh%x%mm<5 0 1 <n 

mzn.ll. i-Jt, VI 0 Ki^Hffl5 0 1 

^ei> i tt\ mm 1 0 2«i . g 1 3 0 2<o 
imp hOL-cmm 5 0 1 mzn.*. ssagsisi t z 

[0067] ^mntmmcmxim-y^ » m 

[0 0 6 8] ARy-A^Sl 04*>feHMP.l 3 0 1 

IWRjWC**. J:->T, HMD1301 
<ft«g|52 1 2t*^-y^*^LTH»Wt«8KL, 
±SStM»«gi52 1 2t*}V^T4«StutHMp 1 
3 0 1 »»K36^V^±«^««««*±||| 



«j£Wffl&Jft«2 1 2*>fcHMD 1 3 0 1CIS, & 

[006 9 ] %m®tiARmm£.tizm£^ 
I. 

[0 0 701 m xmmmizm&ARy-^w 

[007 1 ] *SQtJBJ»«7n-f-v.-MiEl8 
<OXf'!'7S8 04 > Xf77S8 0 5, Xf77"S8 

[0072] -Pi'iJ . HMD 1301 iz$&k?l±m& 
jS A RIWR ( V>"Ciix HMD 13 0 1 (0 

mza\^X\i, XT"/TS8 04, Xr-x7-S8 0 5{i 
JWkft*. tfcfcfc, XT>yrS8 0 4(Ci5V>T, * 

2 0 3**#X? 1 0 
%m&ttY$h. tits X^7-S8 0 5(S».V^, 
*S1BA««^«.2 0 3*s«ffiaSARl«*i4flM- 
&. Xr-y7-S8 0 6^V^ii, £ti££lftft 
MS* 2 0 WWSarijmIm^xtm 1 0 

btz&ui. ±mmm&m2 1 2*\ ±w^€ 

«««ftHMD13.0 1.fcttJ^&. 
[0073] «±W i a 0 fcB 8 Sr^H bfefea, 

[0074] &±eMw>m* ®$®mm*um. 

rn^Aa-HfciOs *^v-x;W-co^^7"<?)H 

MD 1 30 1 ^ffl^fcARy-AttJV^T. 7V-V1 
Oim«OS5H#t*«II^AR««*li^^6C:k*« 

[0075] [»3 nmmmm ami. 2 commmmx 

ii, iitJlQimx^yX-fliUfeZtiX^ 
it. -5^0. *mS^ARI«tfWiil^<xfc^51 0 
3 ^< 5 a- 5 ^< - ^ t: ^"v vfc m« o . * I X 
W7l 0.3»;M9/t?X-*ra?r-*ttA.R.y~A 
^ffl 1 0 4(0rt*c0R AM 7 0 3 t«$n/-cfflk LT 

[0076] ^^51 0 3W*^7 >V}*-9 

<?>oh. \m<n\mtm*7v-*?i o 1 , t u<ti 
7V-- vi o nM<r)m=%mmz>m, mizv 

TWAMz^th^^th* "7t0s 

=5rfc, *5lltff»i3V^fflvM>HMD{i 

^ i vm&jmiz&^xm^tet'rti'-zfr-cofA 

■C* l>HM DJit'T^^-x^-co^^ rtRRg^tut 
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[0077] *^7l 03cO{5B, miVTWJJ* 

1 0 3cofiK, »»*n«M--6^H , c*&trai¥a*ft 
[0078] ®4\tt9mMl»iz&^x<nm<?>&ii* 

I o o 7 9 ] $ l «,ltW;*Wt:fc^ 

Tt, ;M5^5*-*tHI8tfSI»2 0 4tt. HMD 1 

o i m *-i>*mmmi. mmmt i 

X%*yl03CD\H3^7*-9£imiX^l. 
MflBJB#*SS 1 <0llM#ii:ltft£<0(4, *M 5A-;M- 
9tftmEffl 2 0 4 9 1 0 3 (Cllif LfcER^K 

(mwmtmmm) mm&motixfoi, 

[0080] 7"V—Y i o 1 »^^( ^-7i^ 

XteJ:0;M5 1 0 3cO{iB, mz£%ifc)$r&, * 
* ww^l«tlf2 o 4i4*x? i o arott 

a«2 04(Cj:SttiWS»«T-^ (;M510 3tf)ffi 
B, &gW)T-?) fc, 8!Wl^flHBt*&rt»^5^- 

mmmm\mm2 o 2 fcitosfii, . * lt* 

fl^lftft&fttf 2 0 214, Aa3ftfc;M?l 0 3C0 

;M9^9*-#j£*sft>i: , n 1 <om»Bi»ctj^t 

nmk^Xiitd 1 osaBBJBf: PHKT*S . 
[008 1] *fc, 1 0 3<OfirB, *»*£*ffc$ 

HttAR^-AKatl 0 4 CORAM 7 0 3fcH6WStLT 

[ 0 0 8 2 ] 4 fc , *mfflto£&»l 
(408CO7H— f-v-htj^ItT**. /Sfc'U Xf? 

rss o i icfcvvc. *^7/^^-^ltimasi5 2 o 

4(4, HMD1.0 7t*>5 10 3OS^tt«i:je» 
[0083] W±cD^B<Offi$, 

[0084] [ffi4WSaee«] »1 J!;S30ie»»J» 
(c*5iv£(4, a WiHBte3ftTVwarV>. U> 

iCfcT-, T-f XTW 1 0 6t; HifccOffiB, 
^cOgfPMA RWfcfcffcSf £ £ k tfX% h , 

[0085] m9iz*mkmmi l zmz>*y< 7^ta 



[0086] 901a, 90 1b, 901c fifuB, £ 
^5901 a,#^9901b,"*i(?901c)T» 
kRfCftS. 3^c0^^7^nf'tl^^7A"5^ 

-?imi 2^tm^t^xm, zco3^?)t- 

?(4AR?-A§gBl 0 4 CORAM 7 0 3£*tlWU& 

mil. *x$^?^-^n-iawRs»2P4ttJv^ff 

[0087] HI 2fcfeWCl 20 1 , 120 

2, 1 2 0 3i4^iX-e^^5 9 0 1 a, *^5 9 0 1 
b, W?90 1 cWS (HAW*) ffiffl 

[0 088] 9 0 2(4, t/*yCD®mi$iS.X\ *V 

9 9 903 iz x nxmmtifct* yfrhcom ( ^ 
mimtit^m^m^m^m on-*® 

D#x.gB9 0 2£>lLT, WfcittitM2 0 
[0089] 903(4, i^cOflSflgSr^-kk^^T, 

3-?<w 9 fcffiWHGLfc 3-p«#* y (#*y 
a. #?yB, »yo i-<ifejK?y<?3#*» 

i *U99 9 03i&. \?iiX®#*9*Mlkilifr* 

*?mm*, mimw.9 0 2t, a* 

?W»»2 0 4Nitfj.t*. Btt»<0#*VA*ff 
ti/M590la#, #?yBHflt£:*M59 0 1 
b#, yc Srfft a 9 9 0 1c f ftftfitt 

[0 09 0] ^^5A°5X-^$lJW2 0 4(4, -feU? 
* 9 Q 3 *» Afl Ufc # X 5 Oil #M *K t ^^-5 X , 
ij*K:tJV^KWLfc^50^^ (^.1 201,^ 
7'1 2 0 2 , 991 2 0 3 ) fcjBfflU HIRSiX^^ 
7C0^^< 5 ^5^-^ ZZfflteVM&im 2 0 2 K3J 
«il W5901b &ffiffltl.*&{£tt, 
#y y B SriftCl t (,z i '9, ^ ^ 1 2 0 2#ftf£tl># 
^790 1 bW*^?/.^^-**', 

[0091] ^mBmmmcoya^^-hmi 

0(C^-t„ 

[009 2] Xf77S100imitit *lV99 
9 0 Sco^^y^ffSfLTV^^^SrflSt^.. * 

[ 0 0 9 3 ] ZtvTS 1 0 0 2, ^f-y7°S 1 0 0 

3 , XrvTSl 0 04 C*3VVC(4> ylffl S 

nfc^e^^fs ; y b**w 

fc, AT77S 1 0 0 3*»feXf -/TS 1 0 0 6Nt 

[ 0 0 9 4 ] Xf 7 7'S 1 00 5, Xr 7 rs 1 00 
6 , ^f'vrs 1 0 07(CtJ'V)T{4, ff$ix/t*^yt 
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isii9 o 2 iitixmm^m^m 203 ta^n 
*x -?ftsit as 2 0 4 tgfe^xi. . 

[0 0 9 5] ja±W7n-^-htfl£->fcro.^5A 
W3#^«ll9-0 2<0rta!WBPP*<O^t'J 

[0 0 9 6] sSr*J. #iato&JRfc*$vvt 3o^X 50 
* > 5 > - ? liBSS ixT v -eft^tuDtf ^ 

^^5vXtA^AR^-A^S1 0 4t«SHM-4ifc 

[00973 tAhcom-comfo, m^stms^m. 

*«HrtARWfc*?Y*7M 1 06t«wt*it 

[00 983 [« 5 colOI»J»] SS 1 JbS4 DggfflBffi 
fctJWt* «WIBjaARtt«JiT^7-M. 1 0 6fctB 

^tsc:tt% ;<ofa7Hi06mrrv- 
Us/v— V l 0 l ®m IX i J: v \. -3 £ 0 7V-- V 1 

0 1 ITV>& HMD (i^^XJkH<li 

5 0 1 1:, aeteii^f J: ^ 4«*itt*6 0 1 am. * 
zii^wmfoiRMBLtmi-b. mmiza^x, z.<n 
m^mM5oiizm;%tix^&mMm : , 

[00993 ;wa^w«*4ja^6fe*fc:> mmm 

5 0 1 ^^ftS6 0 1 mfei. z<D0?m®6 0 1 

t««tl^ARW«*##a07 , n^5A3-h*&, 121 
aPX.T, AR^-ASIll 04<ORAM7 0 3Wg*W 

x\ m$wmmmm5oiizm^i~z,zttfxz 
*. 

[01003 J^fcJsvvfRBjitfc. **«W6 0 1 t 
*»aSARWfRS1lsatr7-o/5A3-b*07n<- 

1**011 

[01013 77s 1 1 o-ite*»vsTtt, mm& 

6 0 1 *tjyt*^*>o ! F!i»rtff -5 . i*i«6 0 1 i 

KftS^iH^JRJJ, 0 1 #AR?Wa£ 

fes ^ffl^6 0 l^KitS^jftxoavyb'WA^fc: 

[0 1 0 23 ^fvTSl l-0-2ttJV^T|4, 
6 0 1 cOi^ftKOAAiHt 5 . £OAWiflWt*f» 7 0 

5 »4, 1 0 1 mumf-T^ 



AxzmxAljlxbXw 

1 0 1 0 3 3 XfyTS 1 1 0 3 fctJ^TIi, H^P$ 
6 0 l(0^$cOA;j^fd„ i«A^4»fP»7 0 5 
frfcAfrf&.'JUi. 71/-^ 1 0 :1 *fffiiE*f|s taM 
X£fflvv?A*LTfcJ;v\ 

[0 1 04 3 Xr>7-Sl 104K*5WO;t f«ffl 

5 0 It, Xr-y7S 110 2, XT-/7S 1 1 03t 
fe^TKS^tti ^^fM6 0 1 feRttS . 

[0 1 0 5 3 XT77S1 10 5K*SWCM\ *^<KW 

6 0 1 fc; gffiiMW«£J&§IS2 0 3-CT4j«3*Ut* 

*fflMiAR««*flfiB"t&. %co%i$k:mmm£.$L 
[ 0 1 0 6 3 *t$. c\cDM-&mmTj*y^4 1 0 6 
[01073 atcomwcoffifc. m&mmmm^um, 

H(£J: 0 > Tl^-Vl 0 1 fciitfMA 

rbwil smffl^ARttfRwi^ijijirfsit*^* 

[H6(7)HW®] ARr'-AcT)7V— VI 0 2ttttft 

*M^o±«i*MA RtttftfcSttt&as* 1 ** . si 

1 5K#7V~Vt*f LT#7V-^±t3$^>^ 
±«^A B«« t ««t ft fctf>f> A R^- ARI<0rt 
SKflWtfc/Kt. 1 0 3ttH£Sfi-Oi9, * 

^7103, f * xrv-f KMtssaaJiH imw 

[0 1 083 Z<7)mx*l£3Acoyi'~ A ?ift%.b Ujftrt 
i^T3AO#7V-^a, b, cjcW 

LT, HMD 1 0 7 A, 1 0 7B, 1 0 7C, 
«M»^«||52 13A, 2 13B, 2 1" 3.G , .ijfijWW 
fl^«ai52i2A, 2 12B, 2 1 2 C & AR^WvH 

b. c-lif flf^lHM.D 1 0 7 A, 10 7B, 107C 
HMt-ffc. *^W^T'fflV^HMD(it'T5}- 
i/-X^-c7)^-f7CO^OTS>^3t) ii \ dcOHMD{i:3t^ 
^-x;i—«^^7-C'iJ;v\ ^-COlfettt., #HMDfc 

*w I. ^tLWSbsm^mm < i t -cw^-cs i , 

[0 1 -,09] *VtAR^-A#%£W2V #HMD\ 
#±m^Bft««f5, #±TO^f^«t«lC0 
IQ»IMBittsv^ittW.Ufcfc 3 'JOBiJlfc.ffVi, #7V 
— V^LT#7V-^r&]WtC^S^±WMA R 
Wft*ftri/-t*«ttiHMD 1 0,7 A, 1 0 7 
B, XQTC&Wltt.. 

[01103 ±a«SBw«ava^i^« 
mmteyu-^wmv 3 at* set titz h 

<7)X°m< , c:<oc;-k ti±i£<«U .ij^ST** . 
[0 1 I 1 3 «±^aco«w«^M!S® 

>T^i(C.k OSiC07V— Vt^LTiaSAA-RWft 

[01123 Lmcommmmi mcommmmx^A 
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[0 113] U>U 1-0 614— 1t,Z 

7 1 o 3frh<vmm&mim%mmT < 
xrvA i o eizfmxm-r&ztv, -t^xm* 
? i o 3fr^co%mm&ARm&zmmmT<xT\s 

A 1 0 6£J; «)TU-*1 0 im<7)mE.%lztUtlZ 
ttfX'Zh, 

[0114] *mmmx\&mL<?)** ? i o s tm. 

£«T 4 XTW 1 0 6 SriaS-tl»^to^TiEK 
•!>„ HI 6taSfc^^7l 0 3A^^«?MARW 
ft£«ifc&OT xrW 1 0 blz&r^it&tztb^A 
R^'-AgBl 0 4«ftSI5ffii££*-t. 
[0 1 1 53 £«0T143-3iO#;><5 1 03A, 103 
B N 103C£lg£U W?'*U?«M7l03A s 1 
0 3B, 10 3C*^^IS^ARI1!J:«2:«^?»t 
4XTM106A„ 106B, 1 0 6 CSriS^LT^ 
-?■ LT^T-f XTlsA 106 A, 106B, 106 

XTM10 6A, 10 6B, 10 6Cfc*fJELfc*fl 
ttriUftft£&ffi2 0 2 A, 2 0 2 B , 20 2 C, »ffiB 
jSW«&«»203A, 203B, 203C£AR^- 

[0 1 16] -eL-TARy-M^tLiS', 

mmWc&^XWmLlzt -3 9*>«I*frW 

7 1 0 3£:HJSL£#T>fX7''M 1 0 6fc#;M9 1 

0 3 A> ^miA A RWflW*** 8 #i * . 

[ o 1 1 7 3 ±.mmmffi®.&vm-£fm®m 

^-ftUtriXTVA 1 0 6#3GteRR}£3;KfcfcO'C 
143r< , £0£ fcJtiSEoSMHfcJ: OWST**. 

[ o 1 1 8 ] . . utnmmmmvmiiimmm 
mrniz* mmmx? i o 3*>fe«i?MAR» 
*wi»»fa7if>( i oefci'Drw^i 

0 1 Gtfk0ffE#tJl#* £ t **C& 6 . 

[0 119] [m80H»^] 7V— VlOlfcAR 

m&zm-tmmiy-^mmx hxw ■?* 

Do AR^-ASBl'0 4WfU>"5K, flaWttSfcHi 

1 0 mAR8B^<D±il«£ARW£ji&£iT' 

■$-*£fc#r&, •f-rX^M 1 0 6fcl4*«^#& 
S^ARUMRfcU-C^Snrv^., 
[0 1 203 ;£0±IIMARM, WA 

[01213 [Sg9<0H«f#i] 5 1 0 3<Dm& 



^510 3#>£>A:fi-f &H5|WgW> 
8152 0 4 (MltAi) U ?^y*-?1t& 

mmx-wM-tz. x, £oft£0Bw>S£k#*-r 

-nmmm%2 04 ofejetj; 5^9^-^ t 
s$r£*>«i«£itrt-£ tx\ mmzm-t 

[01223 [flmosoteeiB] mcommm^m 

ZbizX^Xh. MmtlZ>Cbm?&Xi>%\\ £ 

[0 1 23 3 *fc. ayKa-^39«K»BLterB^9 
^a- £ fc fe-Js 0 4 J5*t6IO|a0ftnti 

» 3>v*-?±.xMmix\)h$^i>-T 
^ vri/xtk (os) ta btfmmco&mn-® t tmm 
*<nm&s. ^xmuzmmmcom&m 
m^tit^htttiizbmotx^w 
[01243 zuz. tmmw-h^m^^titzruf 

F^n yv^-fizmztitzmmm*-- * b fc«i*> 

*> !» CPU^r ir**HIR«0«SlO-»* £tit£» W \ * 

^fc*4ii6£fctt»d* ; Ct>*v\ 
[0 1 253 JiU;(0^MH^±IBie'ti^*taffl-rS 
^«lB«frti±. jttKWL-^ (HSXfiHl 

[%B^m3 rv-vwiti^tiaARy-Acr, 

C0^#:«^««#O#Sc0»S*^WARy-A«W 
[Kaoffi*3r!M!3 

[H I 3 ^i«nstm^^w*i>„ 

[H23 si commmmmm^m^t^ 7 ^ 

[H3 3 immffMtom* * m&frt>%fcwm 
mm^^i-&mmffim&m-mm*^tmx 
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m 3 commmmcom^mi^i-y'vi 7 ? 



m 1 nm4 vmmt^xri^rm 

H 5 OgWffil t V %T 7V — v tits § tl 

mm^-tmxht. 

[07 ] ARy-.AgacOflgB^'rc 7 ^EI^rS-tBI 
±f»M A RW , mmti A R WfiSr 
^4 cOgllMTffl V M *X7^X7-J±Z* 



[010] ffi4WS®W0BBttJft&7n-f«r-bT 

[Sin ®5mnmmizm&7v—i-*-hx 

[012] m«»wt*3vvc, mm*??) 
fitjjut—p^Ltzmxhh. 

[013] 3t^^-X;P-0^^r(7)HMDSriiBJ!1- 



[014] %2 commmcowMmtii*i-7'v 7 
?0-e$>l>. 

[015] me ommmit^i-mxfo h . 
[016] *7<o3a«B!B*^*-ei-e*s . 
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